The invention relates to a method for providing a model of the surroundings for a vehicle, to a corresponding computer program and a computing apparatus and to a vehicle for the same purpose.
In future, motor vehicles will have an abundance of driver assistance systems which warn the driver of collisions, for example, and possibly also attempt to avoid collisions by means of interventions. Examples of such driver assistance systems are an emergency brake assistant, a lane-keeping assistant, a blind spot assistant, a parking assistant and a so-called automatic cruise control (ACC) assistant, in particular for freeway journeys. In addition, highly automated driving, that is to say the movement of a vehicle without (or substantially without) human intervention, also presupposes knowledge of the surroundings of the vehicle. In order to provide these functions, knowledge of the surroundings of the vehicle is decisive for driver assistance systems. For this purpose, the surroundings are scanned or recorded using one or more sensors such as radar, lidar, camera, ultrasonic sensors or similar sensors known from the prior art. The occupation of the surroundings by objects is then detected with the aid of the sensor measurements with the aid of signal processing methods which are likewise known in the prior art. The occupation indicates that the surroundings cannot be traversed by the vehicle in a particular section and therefore indicates the position of the object. The type of objects is additionally detected, that is to say whether the objects are pedestrians, vehicles, road boundaries, etc. The detected occupation and the types of objects are used to create a model of the surroundings which provides information relating to the occupation of the surroundings by objects, that is to say, in particular, those sections of the surroundings which are occupied by objects, and the type of objects.
One concept for highly automated driving on freeways on the basis of a model of the surroundings is presented, for example, in “A legal safety concept for highly automated driving on highways” by Benoit Vanholme, et al., Intelligent Vehicles Symposium (IV), 2011 IEEE, Jun. 5-9, 2011, pages 563-570. Such a concept is likewise presented in the dissertation “Highly Automated Driving on Highways based on Legal Safety”, University of Evry-Val-d'Esssonne dated Jun. 18, 2012 by Benoit Vanholme. This publication also presents the concept of phantom objects. In this publication, a phantom object is a fictitious object which is assumed at a distance for which no occupation can be created with the aid of the sensors of the vehicle because the measurement range of the sensors has been exceeded.
In addition, when creating a model of the surroundings, the problem exists that no objects can be detected in regions of the surroundings which are concealed by objects in so far as the objects prevent sensor measurements in these regions. The model of the surroundings does not provide any information relating to the occupation by objects for these regions. For example, a stopped bus may conceal a pedestrian in front of the bus. Consequently, no information relating to this pedestrian would be present in the model of the surroundings. If the pedestrian then emerges behind the bus, his occupation will suddenly appear in the model of the surroundings and will cause a possibly severe and sudden reaction by the driver assistance systems. There is also an increased risk of an accident.
An object of the invention is to enable driver assistance systems based on models of the surroundings to take into account possible concealed objects.
In one aspect, a method for providing a model of the surroundings for a vehicle comprises: providing a model of the surroundings which was obtained on the basis of sensor measurements by sensors of the vehicle, the model of the surroundings providing information relating to the occupation of the surroundings by objects and, in particular, the type of objects in the surroundings of the vehicle; determining a region of the surroundings for which no information relating to occupation by objects is provided by the model of the surroundings, the region being within a distance limit for which the model of the surroundings could provide information relating to the occupation by objects on the basis of sensor measurements; checking whether a phantom object needs to be added to the model of the surroundings in the region, in particular using a predefined regulation; if a phantom object needs to be added: determining the occupation by the phantom object in the region of the surroundings, and generating an extended model of the surroundings by adding information relating to the occupation by the phantom object to the model of the surroundings; providing the extended model of the surroundings. The type of object may also relate only to whether the object is moving or is stationary.
A virtual object, a phantom object, is therefore assumed in the model of the surroundings for the concealed region of the surroundings. Phantom objects are, in particular, assumptions of concealed other road users and therefore a safety measure. This assumption compensates for the “blindness” of the sensor systems for concealed regions. Driver assistance systems, in particular systems for highly automated driving, can therefore take the object into account in their planning and function. The precautionary measure of the driver assistance systems is therefore moved to the model of the surroundings by the respective driver assistance system itself in one implementation. The safety measure in the model of the surroundings would therefore have an effect on all driver assistance systems which use the model of the surroundings. In addition, phantom objects can also be added by a driver assistance system or by a component between the creation of the surroundings and the driver assistance system.
As a result of the inclusion of the phantom objects, driver assistance systems can adapt their function in the best possible way to the traffic situation (for example can limit the speed) and can therefore avoid possible accidents or drastically reduce damage. In one typical implementation, a phantom object can be distinguished from a real object (detected using sensor measurements) in the model of the surroundings. The phantom object is therefore indicated as such. Driver assistance systems can now include phantom objects differently than real objects in their function, in particular journey planning or trajectory planning and selection, and therefore do not react to phantom objects to the same extent as they do for real objects. For example, a driver assistance system would not plan any delay in the journey planning for a relatively slow phantom object (even though it would do so for a relatively slow real object) but would nevertheless plan, in the journey planning (for example for future travel trajectories or speeds), that an accident can be avoided or mitigated if it subsequently emerges, on the basis of sensor measurements, that a real object is at the occupation of the phantom object. In an alternative implementation, a probability for its actual existence is assigned to each object in the model of the surroundings. This probability can generally be set lower for phantom objects than for objects which have been detected using sensor measurements.
When checking whether a phantom object can be assumed in a concealed region, it is advantageously assumed that the phantom objects, that is to say pedestrians or cyclists, for example, comply with the traffic rules or at least behave in a predictable manner (and in the process do not comply with the traffic rules). It is assumed, for example, that a pedestrian is on a pedestrian crossing which is concealed by a vehicle. At the same time, a phantom object need not be assumed for each concealed region. For example, a pedestrian is not automatically assumed as a phantom object behind a freeway pillar on a freeway.
In one development, the checking operation comprises the determination of whether an object in the surroundings prevents sensor measurements by sensors of the vehicle in the region at least in such a manner that no information can be provided by the model of the surroundings for this region. The region is therefore concealed by the object for sensor measurements and the object is a concealing object.
In one development, the determination of the occupation by the phantom object in the region of the surroundings comprises: determining the type of preventing object and determining an assumption with respect to the occupation by the phantom object and an assumption with respect to the type of phantom object with the aid of a pre-stored assignment. The assignment may specify rules or regulations regarding where the phantom object should be placed and where its occupation is. The occupation can be oriented to the model of the surroundings and its limits and to the preventing objects. The computing means are set up to use these rules or regulations in order to determine the occupation by the phantom object.
The assignment may specify a rule for the occupation by the phantom object, the rule spatially relating the occupation to the region and/or the object, in particular. A separate rule can be specified for each type of concealing object.
The type of preventing object may also relate to the difference between a stationary object and a moving preventing object. Therefore, only two types may exist in some implementations: a stationary object and a moving preventing object.
In one implementation, the checking operation also takes into account the type of preventing object, in particular using a predetermined list. A phantom object can therefore be assumed for some preventing objects and cannot be assumed for others. If the preventing object is a bus, a phantom object in front of the bus can be assumed. If the preventing object is a bush on a traffic island, for example, no phantom object needs to be assumed behind this bush.
The method can also comprise: determining the type of phantom object to be added and, in particular, its occupation of the surroundings on the basis of the type of preventing object, in particular using a predetermined assignment which comprises the list, in particular. If a bus is the preventing object, for example, a pedestrian can be assumed as the type of phantom object in front of the bus but not a motorcyclist. The type of phantom object can depend on a predetermined assignment. The assignment links the type of object to the type of phantom object. The type of object alone can be understood as meaning a list indicating the types of preventing objects for which phantom objects are assumed. Furthermore, the occupation of the surroundings by the phantom object, that is to say its position, is determined, in particular. The occupation can be determined using the distance limit, with the result that the occupation is positioned precisely on the far side of the limit for which sensor measurements are still available, in other words: precisely within the region or at the boundary of the region (for example 1 m or 0.5 m away from the boundary of the region). The occupation by the phantom object can be specified by means of a pre-stored assignment.
Furthermore, the type of phantom object to be added can be determined on the basis of the type of environment and/or the time of day. For example, in the environment of a school, a (playing) child can be assumed as a phantom object (which moves onto the road from the concealed region) behind a parked and preventing vehicle during the day. Conversely, a pedestrian is not assumed behind a vehicle on a country road at night.
In one preferred development, the checking operation also takes into account whether the traffic rules or predefined further rules applicable in the region provide for other road users to be in the region. The checking aspect with respect to the traffic rules can be carried out on the basis of detected road markings (for example part of a zebra crossing or a road marking and its assumed continuation). The checking aspect with respect to the further rules can be carried out on the basis of predefined regulations. In order to avoid providing too many phantom objects in the extended model of the surroundings, phantom objects are assumed only where they can also be expected in all probability, that is to say only in a concealed region in which the traffic rules or the predefined further rules simultaneously provide for other road users. For example, pedestrians can be expected at a pedestrian crossing concealed by a stationary vehicle or in front of a stationary bus or an automobile can be expected in front of a truck to be overtaken. In this case, other road users cannot only be assumed where the traffic rules provide for this, but also where they are typically situated, that is to say in front of a stationary bus without a pedestrian crossing, for example. In order to ensure a useful method of operation of driver assistance systems, all possible phantom objects should not be included irrespective of traffic rules or other rules, that is to say no collapsing bridges or a group of drivers traveling in the wrong direction on the freeway. The methods of operation of driver assistance systems would otherwise be greatly jeopardized. When selecting the further rules (which go beyond the traffic rules), a balance between defensiveness and assertiveness is advantageously found. Another rule is, for example, that, although the other road user does not comply with the traffic rules, he nevertheless behaves in a predictable manner and is subjectively defensive.
In one development, the method also comprises: if a phantom object needs to be added: determining the direction of movement, a plurality of possible future trajectories of the phantom object and their associated occurrence probabilities and/or the behavior of the phantom object to be added; and supplementing the extended model of the surroundings with information relating to the direction of movement. The direction of movement can be determined using the type of preventing object and the arrangement of the objects on the road. The possible future trajectories and their occurrence probabilities and the behavior can likewise be determined using the type of preventing object and the arrangement of the objects on the road. In this case, it is possible to resort to pre-stored scenarios (typical constellations of types and arrangements). It can therefore be assumed, for example, that a pedestrian concealed by a stopped bus crosses the road. A “worst-case” assumption can always be made for the direction of movement, that is to say the direction of movement is on a collision course with the direction of travel of the vehicle. A driver assistance system would then be set up (also when taking a plurality of trajectories into account) to minimize the risk of collision.
In another aspect, a computer program causes a computer to carry out one of the methods above during the execution of the computer program.
In another aspect, a computing apparatus comprises electronic computing means which are set up to carry out one of the methods above. The electronic computing means may be a computer, a microcontroller or dedicated circuits. The computing apparatus may be caused to carry out the method by a computer program.
In yet another aspect, a motor vehicle comprises sensors for detecting objects in the surroundings of the vehicle and above computing means.
Other objects, advantages and novel features of the present invention will become apparent from the following detailed description of one or more preferred embodiments when considered in conjunction with the accompanying drawing.